Bootstrap semantic PR #21 rebuild - #22
Merged
Merged
Conversation
Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Keep TWMS' historical filesystem cache layout, but teach it the useful cache behavior from Radioxoma's fork: cache_ttl checks, reusable .tne markers, stale fallback after network errors, and atomic replacement for freshly downloaded tiles. Also accept Radioxoma-style dead_tile dictionaries while preserving the legacy dead_tile file comparison. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Keep TWMS' built-in projection shortlist tiny, but make the core WebMercator path tolerate pole-edge bounds and route optional pyproj transforms through the modern Transformer API. This deliberately preserves the existing optional twms[proj] behavior for non-core projections instead of taking Radioxoma's hard pyproj removal. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Harvest Radioxoma's WMS 1.3.0 capability work without taking the fork's API/config rewrite or legacy module removals. Keep the existing WMS 1.1.1 generator intact, add a tiny ElementTree WMS 1.3.0 response, accept case-insensitive request keys, and support CRS:84 as the lon/lat WMS 1.3.0 request alias without advertising it as a WMS 1.1.1 SRS. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Harvest the useful deployment and client guidance from Radioxoma's README without taking the fork framing, unsupported-feature removals, or repo layout rewrite. Document the current upstream entry points, QGIS WMS/WMTS URLs, JOSM TMS/GetTile/file-cache forms, cache behavior, and optional dependencies while keeping TWMS described as a general tiny WMS/tile service. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Preserve the Windows/JOSM single-executable deployment path without adding runtime framework code to TWMS. Build both the stdlib server entry point and the preserved web.py entry point with PyInstaller in GitHub Actions, upload the executables as CI artifacts, and run the workflow for branches, tags, releases, PRs, and manual dispatch. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Do not take Radioxoma's hard removal of filters, corrections, canvas, and reprojection helpers. Instead keep the legacy surfaces covered and fix current Python/Pillow breakage: use a Pillow resampling compatibility helper, keep canvas usable on Python 3, keep GetCorrections returning text/plain, pass WKT colors correctly when no color parameter is provided, and use Python 3 iterator semantics for GPX colors. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Validate downloaded WMS/TMS image bytes before saving them to the filesystem cache, keep stale cached tiles on invalid downloads, and convert mismatched tile responses to the configured layer extension when possible. This keeps the useful cache-safety behavior from Radioxoma's fork without adopting the larger fetcher rewrite. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Record HTTP 404 tile responses as .tne cache misses and allow layer configs to opt in additional HTTP miss statuses with dead_tile.http_status. This keeps the useful HTTP-status TNE behavior from Radioxoma's fork while avoiding service-specific heuristics and noisy response-body logging. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Accept readable tile URL templates with {z}, {x}, {y}, {-y}, and {q} while preserving the legacy percent-template path and transform_tile_number behavior.
This takes the useful URL placeholder idea from Radioxoma's fork without migrating existing configs or adopting the larger fetcher rewrite.
Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Generate a JOSM maps-1.0 imagery XML document from configured TWMS layers and expose it through both request=GetJOSMImagery and stdlib routes at /josm/imagery.xml and /maps.xml. This keeps the useful JOSM imagery-list idea from Radioxoma's fork without taking the heavy route/config reshuffle or launching JOSM remote-control side effects. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Pass the target image format explicitly when saving through a temporary cache path, so Pillow does not infer the format from the .tmp.<pid> suffix. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Pass optional per-layer HTTP headers to tile and WMS upstream downloads while leaving layers without headers on the legacy string-url path. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Honor per-layer min_zoom in both tile and WMS fetchers while preserving the existing exclusive max_zoom behavior. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Add an opt-in cache_layout for slippy/MOBAC-style <z>/<x>/<y> cache paths while keeping TWMS' historical grouped cache layout as the default. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Allow upstream WMS remote_url templates to supply {bbox}, {width}, {height}, and {proj} while preserving the legacy base-URL append behavior when no WMS placeholders are present.
Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Keep the stdlib threaded server tiny, but avoid waiting forever on a stalled upstream tile or WMS source. Layer configs may override the timeout when an old deployment deliberately needs different behavior. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Keep malformed REST-style protocol paths from falling through to the overview or disconnecting handler threads. Also fix the legacy GetTile fast-cache layer lookup exposed by WMTS REST tile requests with query strings. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Keep the preserved WmsCanvas helper from waiting forever on remote WMS tiles while retaining an explicit timeout=None escape hatch for deployments that relied on the historical unbounded wait. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Preserve the legacy overlay/filter surface by making configured empty_color values transparent again, including per-channel empty_color_delta tolerance. This repairs the behavior instead of deleting the old rendering helpers. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Use integer cache shard directories under Python 3 and read cached GetTile responses as bytes, so the historical grouped cache layout remains usable alongside the new opt-in zxy layout. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Preserve the legacy cache_tile_responses fast path under Python 3 by reading and writing cached image responses as bytes instead of text. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Harvest Radioxoma's HTTP retry idea without taking the larger backend rewrite: keep urllib, default to one attempt for legacy behavior, and let global or per-layer config opt in retries for transient transport errors. HTTP errors such as 404 still flow to the existing TNE/cache handling instead of being retried as generic network failures. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Preserve the legacy max_ram_cached_tiles cache, but make eviction least-recently-used instead of FIFO/off-by-one history popping. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Keep legacy cache_tile_responses configs working when they use MIME strings such as image/png, as shown in the packaged example, while still accepting the normalized Pillow format keys used internally. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Document the decision not to import Radioxoma's Firefox cookie-store discovery into upstream TWMS. Private deployments can still provide cookies through per-layer headers or their own Python config code without adding a browser-profile dependency to the tiny server. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Keep the useful hygiene from Radioxoma's added ignore file, but reduce it to artifacts TWMS actually produces during tests, builds, and PyInstaller packaging. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Accept Radioxoma's readable bounds layer metadata key as an alias for historical data_bounding_box/bbox in tile filtering and generated TileJSON/WMTS metadata without replacing the legacy config contract. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Harvest Radioxoma's useful JOSM imagery metadata idea without importing the larger API/config rewrite or launching JOSM remote-control side effects. The generated maps-1.0 XML now carries layer bounds, overlays, attribution URLs, valid-georeference/default hints, and fixed per-MD5 no-tile checksums while preserving TWMS' existing URL and zoom semantics. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Keep the legacy overview page but harvest Radioxoma's useful layer metadata behavior: accept the readable bounds alias for preview/bbox links and link configured provider URLs without reshaping the overview UI. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Preserve the small launcher deployment idea from Radioxoma's branch without installing desktop integration by default. Package Windows batch and Linux desktop-entry templates under share/twms/contrib and document them as optional manual deployment helpers. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Keep the CI queue useful while the PR is being rebuilt commit-by-commit. Cancel older runs for the same workflow event and branch, but keep push and pull_request groups separate so the latest checks remain visible.
Treat a layer mimetype as the source for its tile extension when ext is not configured, and normalize legacy ext-only configs back to MIME types at config load time. Keep existing ext configs working while letting TileJSON, WMTS, JOSM imagery, overview HTML, GetTile cache paths, and downloaded cache writes use mimetype-only layers. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Komzpa
force-pushed
the
ai/pr21-semantic-rebuild
branch
from
July 26, 2026 12:37
592f9bd to
a86e856
Compare
Accept image/webp as a normal WMS GetMap output format and advertise it in the legacy WMS 1.1.1 capabilities response. Keep the change tiny: it reuses Pillow's WEBP encoder through the existing format map instead of adding a new protocol or rendering path. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Accept an optional layer_defaults mapping from Python configs and expose it through a tiny dict wrapper rather than importing the fork's full config rewrite. Defaults participate in get() and [] lookups while explicit layer keys still remain distinguishable, and format metadata is normalized for defaults as well as per-layer values. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Keep the stdlib server compatible with the useful route shape from the fork by accepting /wms/<layer>/<z>/<x>/<y> tile URLs, /wms query requests with a /wms service URL, and /josm/maps.xml. This keeps the route layer tiny and delegates behavior to the existing twms_main requests instead of importing the fork's larger api.py split. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
List the useful local overview, WMS, WMTS, and JOSM imagery URLs when the stdlib server starts, and expose the TWMS version in the Server header. This harvests the helpful operator-facing part of the fork's server startup changes without importing its logging/color-output rewrite. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Allow readable config fetch values such as tms, tile, and wms to resolve to the existing legacy fetcher functions. This harvests the useful config readability part of the fork without importing its TileFetcher class rewrite or provider-specific dynamic URL discovery. Co-authored-by: Eugene Dvoretsky <radioxoma@gmail.com>
Use python -c for the import smoke check so the same workflow step works under both Bash on Linux and PowerShell on Windows.
Komzpa
marked this pull request as ready for review
July 26, 2026 14:04
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This draft rebuilds the useful parts of #21 as a clean semantic stack on top of Komzpa/twms master. It intentionally keeps TWMS tiny and general-purpose: no broad service-specific defaults, no large framework rewrite, no license/author deletion, and no Java-factory-shaped architecture.
Current commits:
88cfbb9bootstraps packaging and GitHub Actions CI.1320671adds legacy package smoke tests and compatibility shims.47455declears Python 3 syntax warnings without changing behavior.996ce0badds a no-network legacyGetTilesmoke test.60bd619adds a tiny stdlibThreadingHTTPServerentry point while keepingweb.py/WSGI available astwms-webpy.f121d79exposes TileJSON-style tileset metadata.1ddb2bbexposes WMTS 1.0.0 capabilities plus REST and KVP tile access, with Radioxoma authorship preserved.3989521modernizes the filesystem tile cache with TTL,.tne, stale fallback, atomic replacement, and Radioxoma-styledead_tiledictionaries.c7c3b49modernizes projection fallback with pole-edge clamping and optionalpyproj.Transformer.317dc88adds tiny WMS 1.3.0 andCRS:84support for QGIS-style clients without replacing WMS 1.1.1.4aa6b65rewrites README client/deployment docs for stdlib,web.py, QGIS, JOSM, WMTS, TileJSON, GetTile, shared caches, and optional dependencies.daed6f6adds a Windows executable artifact job fortwms.exeandtwms-webpy.exe.35c847epreserves filters, corrections, drawing, canvas, and reprojection surfaces with current Python/Pillow fixes.c24f53dvalidates downloaded image bytes before caching and converts mismatched tile responses to the configured layer extension.237dac5records HTTP 404 and configureddead_tile.http_statusvalues as.tnecache misses.2aa87ceaccepts readable named tile URL placeholders while preserving legacy%stemplates.2980eaagenerates JOSM maps-1.0 imagery XML.041580cfixes atomic WMS cache writes by passing the intended image format explicitly.61df8a7supports optional per-layer HTTP headers for tile and WMS upstreams.a1339faenforces per-layermin_zoomwhile preserving existing exclusivemax_zoom.7852d8aadds opt-incache_layout: "zxy"while keeping TWMS' historical grouped cache layout by default.98c39bdaccepts readable named WMS URL placeholders while preserving legacy base-URL query appending.794940bbounds upstream tile/WMS fetch waits with global/per-layer timeout config.faadb4ehardens stdlib REST routing and WMTS REST URLs with query strings.2fe1b21applies bounded upstream waits to the preserved legacyWmsCanvas.ad5ead4restores legacyempty_coloroverlay transparency andempty_color_delta.637ef8bkeeps historical cache lookups and cachedGetTilebytes binary-safe.eb261e0keeps legacycache_tile_responsesbinary-safe.a4e0d9eadds optional transient upstream retries without retrying HTTP/TNE statuses.618a447makes the RAM tile cache least-recently-used and fixes its bound.0dc7b06accepts MIME format keys such asimage/pngincache_tile_responses.d99e15adocuments the cookie decision: per-layer headers yes, automatic browser cookie discovery no.a85b8b4keeps generated-artifact ignore hygiene without broad unrelated IDE/tooling ignores.952f1aeaccepts readable layerboundsmetadata aliases.f168719enriches JOSM imagery XML with bounds, overlay/default/georeference hints, attribution URLs, zoom limits, and no-tile checksums.b29d80daligns the overview page with layer bounds aliases and provider URLs.edabf43ships optional Windows batch and Linux desktop-entry launcher templates as packaged contrib files.b8b3d18cancels stale CI runs for the same workflow event and branch.a86e856accepts layermimetypemetadata as a readable alias/source for tile extensions while preserving legacyextconfigs.f2522ccsupportsimage/webpas a normal WMS output format while reusing the existing Pillow format path.d552b8asupports optionallayer_defaultsin Python configs through a tiny dict wrapper rather than importing the fork config rewrite.d0b1ed1adds tiny stdlib route aliases for/wmstile/query URLs and/josm/maps.xmlwhile keeping routing delegated totwms_main.2cd346cprints useful stdlib-server startup URLs and includes the TWMS version in the HTTPServerheader without importing the fork logging rewrite.fbf33d1accepts readable string fetcher aliases such as"tms"and"wms"while resolving them back to the existing legacy fetcher functions.7e4d2edmakes the import smoke check shell-portable so the same GitHub Actions workflow passes under Bash on Linux and PowerShell on Windows.Branch behavior
The branch currently:
web.pyentry points;pyproject.toml;0.07z, while exposing PEP 440 metadata as0.7+z;imploader withimportlib;GetTilebecause it is still useful for passing filters through tile URLs, including direct/wms/<layer>/<z>/<x>/<y>route aliases;pyprojfor configured non-core projections;.tne,dead_tile, binary-safe response caches, layer headers, min zoom, timeouts, retries, and named upstream URL placeholders;mimetypemetadata as an alias/source for tile extensions while preserving existingextconfigs;layer_defaultsfor shared layer metadata without merging defaults into every layer;fetchaliases"tms"/"tile"and"wms"while preserving callable legacy fetchers;/josm/imagery.xml,/josm/maps.xml, and/maps.xmlas generated imagery-list aliases;Serverheader;web.pydeployment surfaces;COPYINGand README for modernization work;Validation
Clean-venv validation passed through current head
7e4d2ed2bf7ed793d040335f0a2af672a7150154:python -m compileall -q setup.py index.py tools twms teststwms.__version__ == "0.07z"and installed package metadata is0.7+zpython -m unittest discover -s tests -v(75tests,1optionalpyprojskip)python -m buildcontrib/twms.batandcontrib/twms.desktopgit diff --checkThe optional projection extra was also checked separately:
pip install -e '.[proj]'python -m unittest tests.test_legacy_smoke.LegacySmokeTest.test_optional_pyproj_projection_uses_modern_transformer -vThe smoke suite covers WMS 1.1.1/1.3.0, CRS:84, GetTile, stdlib serving, stdlib startup banner, versioned HTTP
Serverheader, TileJSON, WMTS capabilities/KVP/REST, JOSM imagery XML, overview HTML, cache TTL/TNE/stale/dead-tile behavior, binary cache reads/writes, layer headers, min/max zoom, zxy cache layout, named URL placeholders, upstream timeout/retry behavior, projection fallback, optional pyproj, legacy rendering helpers, layer bounds aliases, layer mimetype aliases, layer defaults, string fetcher aliases, WebP WMS output, stdlib WMS/JOSM route aliases, and Windows/package artifact contents.GitHub Actions passed on current head
7e4d2ed2bf7ed793d040335f0a2af672a7150154:Both runs passed Ubuntu and Windows package smoke jobs for Python 3.11 and 3.13, plus the Windows executable artifact job that builds and uploads
twms.exeandtwms-webpy.exe.Selection notes
This branch deliberately rebuilds #21 as semantic pieces instead of merging the fork wholesale.
Taken, but reshaped to fit upstream TWMS:
web.py/WSGI deployment path;pyprojfor non-core projections;boundsandmimetype;api.pysplit;Serverheader without importing the fork logging/color-output rewrite;Not taken from #21:
web.pydeployment surfaces;submit(...).result();Scope
Ready for review. The reciprocal Radioxoma branch update has been posted as radioxoma#1; that fork PR is expected to be a replace/rebase PR rather than a clean merge because the fork branch history diverged from upstream.